Nonparametric Bayesian Models for Machine Learning

Nonparametric Bayesian Models for Machine Learning by Romain Jean Thibaux Doctor of Philosophy in Computer Science and the Designated Emphasis in Communication, Computation and Statistics University of California, Berkeley Professor Michael I. Jordan, Chair This thesis presents general techiques for inference in various nonparametric Bayesian models, furthers our understanding of the stochastic processes at the core of these models, and develops new models of data based on these findings. In particular, we develop new Monte Carlo algorithms for Dirichlet process mixtures based on a general framework. We extend the vocabulary of processes used for nonparametric Bayesian models by proving many properties of beta and gamma processes. In particular, we show how to perform probabilistic inference in hierarchies of beta and gamma processes, and how this naturally leads to improvements to the well known näıve Bayes algorithm. We demonstrate the robustness and speed of the resulting methods by applying it to a classification task with 1 million training samples and 40,000 classes. Professor Michael I. Jordan, Chair Date